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Abstract Body 


Background / Context: 

A positive school climate is characterized by a supportive, orderly, and fair interpersonal 
climate (Anderson, 1982; Cohen, McCabe, Michelli, & Pickeral, 2009; Haynes, Emmons, & 
Ben-Avie, 1997). Children’s perceptions of interpersonal climate and school safety are 
associated with a several academic and behavioral adjustment outcomes (Brookmeyer et ah, 
2006; Bond et ah, 2007; Fisher & Fraser, 1991; Kuperminc, Leadbeater, & Blatt, 2001; Loukas 
& Murphy, 2007). Moreover, a lack of teacher support and affiliation has been linked to teacher 
burnout, lower professional commitment, and lower self-efficacy in eliciting help (Brouwers et 
ah, 2001; Donnan, 2003; Osterman, 2000; Maslach, 1999), all of which have important 
consequences for children’s opportunities to succeed. Most empirical studies have looked at only 
one or two dimensions of school climate, only at one level, or from only one reporter, failing to 
empirically capture what theory would suggest is a multidimensional, multilevel, and systemic 
construct The question of how cross-dimension and cross-level interactions (e.g., contextual 
features of classrooms and children’s feelings of safety) contribute to children’s school 
adjustment remains. Furthermore, with a few exceptions (e.g., Bradshaw, Koth, Thornton, & 
Leaf, 2009), many whole-school interventions seeking to achieve systemic change do not 
quantitatively measure whether they were successful at doing so (Durlak et al., 2007). Similarly, 
while a positive school climate can aid in the implementation of whole-school interventions 
(Bradshaw et al., 2009; Durlak & DuPre, 2008), little research has sought to understand whether 
school climate moderates the impacts of the intervention on student outcomes. 

Purpose / Objective / Research Question / Focus of Study: 

The current study has two goals: (1) to better understand the contribution of school 
interpersonal climate to children’s school success; (2) to examine whether the impact of the 
whole-school implementation of social and character development (SACD) programs are 
moderated by interpersonal climate. The study asks: (1) Is there a direct relationship between 
school interpersonal climate at the start of 3 rd grade as perceived by children and teachers and 
modeled at the individual and school levels and children’s school engagement and academic 
competence/motivation at the end of 5 th grade? (2) Do school-aggregated child and teacher 
perceptions of interpersonal climate moderate the relationship between children’s individual 
perceptions of climate and the outcomes? (3) Is there an impact of SACD programs on 
engagement and academic competence? (4) Do school-aggregated child and teacher perceptions 
of interpersonal climate moderate the impact of SACD programs on school engagement and 
academic competence? (See Figure 1 for a conceptual model.) 

Setting: 

Data come from a three-year, multi-site, school-randomized evaluation of Social and 
Character Development (SACD) Programs in 83 elementary schools in 7 sites and 6 states 
funded by the Institute of Education Sciences and Centers for Disease Control (see IES NCER 
report on SACD Programs, http://ies.ed.gov/ncer/pubs/201 12001/) . 

Population / Participants / Subjects: 

The student sample consisted of 6,567 children who were in the third-grade at the start of 
the evaluation. Children who entered the study schools over the course of the three years of the 
evaluation were added to the sample, and leavers were not tracked after they left a study school. 
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In spring of 2007, the child sample consisted of 6,249 children (69% of the original sample). For 
the purposes of this study, only students who were in the study at baseline were included in the 
sample. The current child sample therefore consisted of 4,245 children in 320 classrooms at 
baseline. Just over half (52%) of the children were girls. Children were ethnically diverse (43% 
White, 30% Black/ African American, 19% Hispanic/Latino, 7% other). The average age was 8.5 
(SD = .47). About 43% of households had incomes at or lower than 135% of the federal poverty 
level. The teacher sample consisted of third, fourth, and fifth grade teachers in the participating 
schools (N = 841 at baseline; 89% female). The majority of teachers (73%) self-reported as non- 
Hispanic White, 17% as Black or African-American, 6% as Hispanic, and the remaining teachers 
as other (i.e., Asian, American-Indian, or Alaskan Native). Teachers averaged 10 years of 
teaching experience overall (SD=9) and 5 years at their current school (SD=6). 

Intervention / Program / Practice: 

The Institute of Education Sciences (IES) and the Division of Violence Prevention in the 
National Center for Injury Prevention and Control, Centers for Disease Control and Prevention 
(CDC) collaborated to conduct a rigorous impact evaluation of universal, school-based programs 
aimed at improving elementary school students’ school adjustment (Social and Character 
Development Research Consortium, 2010). The seven programs evaluated were tenned Social 
and Character Development (SACD) programs. Each of the programs sought to foster the 
development of social competencies, reduce negative behaviors, and improve academic 
perfonnance. Each of the programs used a universal approach, implementing the programs in all 
elementary school classrooms. The programs employed activities to promote six SACD goals 
(character education, violence prevention and peace promotion, social and emotional 
development, tolerance and diversity, risk prevention and health promotion, and civic 
responsibility and community service) as well as behavior management. The seven programs 
were: Academic and Behavioral Competencies, Competence Support, Love in a Big World, 
Positive Action, Promoting Alternative Thinking Strategies, 4Rs, and Second Step. 

Research Design: 

In each site, a research team randomized 10 to 18 schools to the business-as-usual 
approach to implement the SACD program (Social and Character Development Research 
Consortium, 2010). A total of 84 schools were recruited into the study. The randomization 
employed a matched pair design with stratified sampling. Each site created five to seven pairs of 
schools. Pairs were created by minimizing the distance between several measurable 
characteristics for schools within each pair. The school characteristics used to create the pairs 
differed across sites, depending on what data were available. One school within each pair was 
randomly assigned to the treatment group and the other was assigned to the control group. 
Although the intervention was implemented school-wide, the evaluation consisted of one cohort 
of students who was followed over 3 years from 3rd to 5th grade. As part of the evaluation, a 
common set of student outcomes including social and emotional competence, behavior, 
academics, and perceptions of school climate were collected across sites. 

Data Collection and Analysis: 

The first wave of data was collected on consenting students in the third grade during fall 
of 2004. Follow-up data were collected in spring of Year 1 (2005), fall and spring of Year 2 
(2005-2006), and spring of Year 3 (2007) when the students were in fifth grade. Data included 
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survey data from students, primary caregivers, teachers, and interview responses from principals. 
In addition, each site collected its own set of site-specific data. 

Three-level hierarchical linear modeling (HLM 6.02, Raudenbush & Bryk, 2002) with 
fixed effects in which children were nested in teachers and schools was used to account for the 
multilevel nature of our data. Because students changed classrooms across the three years of the 
study, no hypotheses were made about relationships between classroom-level (Level 2) 
predictors and the outcomes. Modeling this level allowed us to include teacher demographic 
controls and teachers’ individual perceptions of climate (also entered as school aggregates at 
Level 3) at the appropriate level and to control for variation in individual teacher reports of 
interpersonal climate. All predictor variables were measured in fall of Year 1, at baseline, and 
outcomes were measured in spring of Year 3, at the end of the study. All models controlled for 
baseline levels of the outcomes. The following measures were included. 

Child-level school interpersonal climate. Children were asked to report on the 
interpersonal climate of their school (1 = disagree a lot to 4 = agree a lot). Negative interpersonal 
climate included 4 negatively worded items from the Sense of School as Community measure 
(Roberts, Horn, & Battistich, 1995). Feeling Afraid (IES/CDC developed) included 4 items about 
how afraid children are that they or other students will be bullied, hurt, or teased. 

Teacher-level interpersonal climate. Three dimensions of teachers’ reports of the 
organizational and interpersonal environment of their schools — teacher affiliation, student 
respect, and school safety — were used (SLEQ; Fisher & Fraser, 1991). The three dimensions 
included 19 items on a 5-point rating scale (l=strongly disagree to 5=strongly agree). 

School-level interpersonal climate. The two child-level dimensions and one additional 
child-reported dimension, peer aggression (Orpinas & Frankowski, 2001), were aggregated to the 
school level and combined to fonn one factor based on multilevel factor analysis, representing 
children’s reports of school negative interpersonal climate. Three dimensions of teachers’ 
individual perceptions of interpersonal climate were aggregated to the school level. Based on 
multilevel factor analysis, student respect and school safety were combined to form one factor. 

Outcome measures. Children rated their level of behavioral engagement (Furrer & 
Skinner, 2003). Academic competence/motivation (IES/CDC developed; adapted from Gresham 
& Elliott, 1990 and Achenbach, 1991) was assessed with 4 items asking teachers to report on 
their students’ performance in math and reading and overall and their motivation to succeed. 

Child-level covariates. Child-level covariates included parent reports of their children’s 
gender, race/ethnicity, and household income (l=below 135% of poverty level). Teacher-level 
covariates included teacher-reported race/ethnicity, gender, and years of experience. School-level 
covariates included treatment status, 6 site dummies, and the proportion of minority students and 
poor students in school. These were computed from parent-reported race/ethnicity and income 
using sample students, and were treated as proxies for school composition. 

Findings / Results: 

Results without (Model 1) and with interactions (Model 2) are shown in Table 1. 

Child level. Children’s individual perceptions of negative interpersonal climate were 
significantly related to engagement (P = -.04, p < .01) and to academic competence/motivation (P 
= -.06, p < .01), but children’s perceptions of feeling afraid were not, after controlling for fall 
levels of the outcome and the demographic covariates. 

School level. Aggregated teachers’ perceptions of teacher affiliation were significantly 
related to lower academic competence/motivation ((> = -.27, p < .01) and marginally related to 


SREE Spring 2015 Conference Abstract Template 


3 



lower engagement (fi = .08, p < . 10). Aggregated teacher perceptions of student respect/safety 
were significantly related to higher academic competence/motivation (/> = .29, p < .01). Contrary 
to expectations, schools in which teachers reported more affiliation had students with lower 
academic competence/motivation. Aggregated child perceptions of negative interpersonal 
climate were not significantly related to either outcome. 

Cross-level interactions. The moderating effect of school-level climate was examined 
by including cross-level interactions between Level 1 negative interpersonal climate and feeling 
afraid and each of the Level 3 dimensions of climate. A cross-level interaction between Level 3 
aggregated child perceptions of negative interpersonal climate and children’s individual 
perceptions was significant for engagement (P = . 13, p < .05). The interaction indicated that the 
negative relationship between children’s perceptions of negative climate and engagement was 
stronger in schools with a less negative climate (see Figure 2). 

Impact of SACD and moderating role of interpersonal climate. Preliminary analyses 
suggest that, overall, SACD programs did not impact engagement or academic competence (see 
Model 1 for each outcome). There was significant moderation of children’s aggregated 
perceptions of negative interpersonal climate (P = .26, p < .01) (see Model 2). Contrary to 
expectations, treatment schools with a more negative climate had students who were more 
engaged, compared to treatment schools with a less negative climate, but the opposite was true in 
control schools (see Figure 3). 

Conclusions: 

Children who experienced a lack of community were less engaged and had more 
difficulty academically, pointing to the importance of children’s own experiences in their 
academic success. Although school-level relationships were weak, teachers’ perceptions of 
support and safety in school were also related to academic outcomes, although the 
counterintuitive negative relationship between school-aggregated teacher affiliation and 
academic competence/motivation warrants further unpacking. As these were both teacher 
reports, perhaps schools with teachers reporting more teacher affiliation also had teachers rating 
children’s academic competence more strictly. The one cross-level interaction pointed to the 
importance of considering the fit between the child and the school environment. Children who 
had a good fit (i.e., perceived a more negative climate in the context of a school with a more 
negative climate) actually reported being more engaged, compared to children who had poor fit 
(i.e., perceived a negative climate in the context of a less negative climate). This finding is 
consistent with person-environment fit theory (Bellmore, Witkow, Graham, & Juvonen, 2004) 
and suggests that individual perceptions and the school ecology should be considered 
simultaneously. Finally, SACD programs did not directly improve academic outcomes. The one 
moderating effect suggests that while SACD schools had lower engagement overall, being in a 
more negative climate initially actually helped boost the impact of SACD programs on children’s 
engagement. In follow-up analyses we will explore the possibility that these treatment schools, 
with more room to improve, showed enhancements in the climate and, in turn, engagement. 

Limitations of this study include low intraclass correlations and reliabilities at the school 
levels for the study outcomes, suggesting low power to detect group-level effects. Further, other 
aspects of school climate such as strong leadership and high organization may act as more 
powerful moderators. Nevertheless, one implication of the findings is that intervention efforts 
that target the school climate need to be attentive to children’s individual experiences, context- 
level factors and to the negative effects of dissonance between children and the social context. 
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Figure 1 . Conceptual model 
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Table 1. Hierarchical Linear Models Predicting Academic Competence and School Engagement 



Academic competence 


School engagement 


Model 1 


Model 2 


Model 1 


Model 2 


Estimate 

S.E. 

Estimate 

S.E. 

Estimate 

S.E. 

Estimate 

S.E. 

Child (Level 1) 









Engagement/academic competence - baseline 

0 79 *** 

0.02 

0 72 *** 

0.02 

0.13 *** 

0.02 

0.13 *** 

0.02 

Sex (l=gjrl) 

0.09 ** 

0.03 

0.09 ** 

0.03 

0.18 *** 

0.02 

0.18 *** 

0.02 

Poor (l=below 135% poor) 

- 0.11 ** 

0.04 

- 0.11 ** 

0.04 

- 0.02 

0.03 

-0.03 

0.03 

Black 

- 0.11 * 

0.05 

- 0.11 * 

0.05 

-0.05 

0.04 

-0.04 

0.04 

Hispanic 

-0.07 

0.05 

-0.06 

0.05 

-0.08 ' 

0.04 

-0.07 

0.04 

Other 

-0.04 

0.06 

-0.04 

0.06 

0.03 

0.04 

0.03 

0.04 

Feeling afraid 

0.00 

0.01 

0.00 

0.01 

0.00 

0.01 

0.00 

0.01 

Negative interpersonal climate 

-0.06 ** 

0.02 

-0.06 ** 

0.02 

-0.04 ** 

0.02 

-0.05 ** 

0.02 

Teacher/classroom (Level 2) 









Staff affiliation 

0.04 

0.04 

0.04 

0.04 

0.00 

0.02 

0.00 

0.02 

Student respect 

- 0.10 ** 

0.04 

- 0.12 ** 

0.04 

- 0.01 

0.02 

- 0.01 

0.02 

School safety 

-0.04 

0.03 

-0.04 

0.03 

0.00 

0.02 

0.00 

0.02 

Sex (l=female) 

0.08 

0.09 

0.08 

0.09 

0.07 

0.05 

0.06 

0.05 

Black 

0.09 

0.07 

0.11 

0.07 

- 0.01 

0.03 

- 0.02 

0.03 

Hispanic 

- 0.02 

0.11 

- 0.01 

0.11 

-0.03 

0.06 

-0.04 

0.05 

Other 

0.32 ** 

0.12 

0.32 * 

0.12 

0.02 

0.08 

0.02 

0.08 

Years of teaching experience 

0.00 

0.00 

0.00 

0.00 

0.00 ' 

0.00 

0.00 ' 

0.00 

School (Level 3) 









Negative interpersonal climate 

-0.28 

0.20 

-0.19 

0.22 

0.01 

0.11 

- 0.11 

0.13 

Teacher affiliation 

-0.27 ** 

0.08 

-0.35 * 

0.13 

0.08 ' 

0.04 

0.12 

0.08 

Student respect/safety 

0.29 ** 

0.09 

0.34 * 

0.15 

-0.07 

0.05 

-0.13 

0.09 

Treatment (l=treatment, 0=control) 

-0.06 

0.04 

0.27 

0.33 

- 0.02 

0.02 

-0.39 * 

0.15 

Percent poor 

- 0.10 

0.18 

- 0.12 

0.19 

-0.27 * 

0.11 

-0.26 * 

0.11 

Percent minority 

0.25 

0.16 

0.34 * 

0.17 

0.21 * 

0.08 

0.21 * 

0.07 

Tx X negative social climate 



-0.25 

0.25 



0.26 * 

0.12 

Tx X student respect/safety 



- 0.02 

0.17 



0.08 

0.10 

Tx X teacher affiliation 



0.11 

0.15 



-0.04 

0.08 

Cross-level interactions 









Negative interpersonal climate (LlxL3) 







0.13 * 

0.06 

Random effects: 









Classroom-level variance 

0.030 *** 


0.030 *** 


0.003 ** 


0.001 * 


School- level variance 

0.011 *** 


0.009 *** 


0.000 


0.001 


Negative interpersonal climate slope 







0.003 


Residual variance 

0.449 


0.449 


0.237 


0.235 


Fit statistics: 









AIC 

5363.95 


5359.38 


3505.65 


3493.08 



Note . ' p <.10. * p < .05. ** p < .01. ***p <.001. 

Child N = 4,050; Classroom N = 305; School N = 84 
Site dummies are included but not shown. 

Unstandardized regression coefficients and robust standard errors are shown. 

The following cross-level interactions were not significant and were taken out of the final models above: negative interpersonal 
climate by teacher affiliation and student/respect safety, feeling afraid by teacher affiliation, student/respect safety, negative 
interpersonal climate (L3). 
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Figure 2. Interaction of individual perceptions of negative interpersonal climate (LI) by 
aggregated perceptions of negative interpersonal climate (L3) on school engagement. High 
interpersonal climate (L3) = upper quartile. Low interpersonal climate (L3) = lower quartile. 


SREE Spring 2015 Conference Abstract Template 


B-2 



Figure 3. Interaction of treatment by aggregated perceptions of negative interpersonal climate 
(L3) on school engagement. High interpersonal climate (L3) = upper quartile. Low interpersonal 
climate (L3) = lower quartile. 
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